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1. Based upon the outcomes outlined in your proposal, DHS expects to implement new technology and 
systems which will improve processes and client outcomes through development of a client portal, a 
provider portal, and predictive analytic tools. How successful was the organization at achieving this 
outcome? 

The grant was used to support two technology innovations: 1) development and implementation of 
Client View, an online portal that provides integrated client service data to DHS staff, providers and 
clients and 2) development and implementation of the Allegheny Family Screening Tool (AFST), a 
predictive risk model designed to improve child welfare call screening decision-making. Responses 
are provided individually for each. 

Client View (provider and client portals): Client View supports DHS's ongoing mission to improve 
care for Allegheny County residents through the collection and provision of a more comprehensive 
human services client record used to integrate and coordinate care across human services. So 
ultimately, DHS has built a tool that serves as a model for other counties across the nation, 
demonstrating the value of integrated data to support the provision of services to vulnerable 
individuals. As of March 2018, DHS has developed and implemented the core functionality of Client 
View, and also completed a development initiative to further enhance functionality for its users. 

The Client View application is available to DHS front-line staff, to support their daily service activities 
and service coordination for clients, and administrators who oversee and manage these programs. 

It further provides direct access to client information for DHS-contracted service providers, giving 
them the ability to access comprehensive information about the clients they serve. 

Broadly, Client View includes information about the following areas of DHS service: 

• Homeless and Housing Supports 

• Mental Health Services - county and managed care-funded 

• Child Welfare Services for both parents and children, including prevention and diversion 
services 

• Aging Services 

• Early Intervention 
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• Independent Living Services for Young 
Adults 

• Criminal Justice Re-entry Services 

• Medical Assistance Transportation 

• Intellectual Disability 

• Family Support Centers 

• Head Start 

Note: Because of D&A regulations, Drug and Alcohol service 
information is available to DHS staff but not providers. 

It also includes information about the following 
external non-DHS programs: 

• Juvenile Probation 

• Allegheny County Jail 

• Allegheny County and Pittsburgh Housing Authorities 

• School Enrollment (Approximately 14 districts including Pittsburgh Public Schools) 

• Medical Examiner 

• Public Benefits - SNAP, TANF, SSI, Other Medical (Commonwealth of PA) 

• Birth Records (Allegheny County Health Department) 

We continue to add to the list of data sources integrated within our Data Warehouse and displayed 
within Client View. We recently integrated 9-1-1 emergency call data into Client View, and 
anticipate receiving employment and unemployment compensation data from the PA Department 
of Labor and Industry. 


For users, Client View is conveniently 
linked to DHS's transactional source 
systems. Staff and providers can click 
on hyperlinked client IDs within those 
systems to be seamlessly directed into 
that client's record without having to 
re-enter their password, manually 
open the application, or search for the 
desired client. 

Within Client View, they can view a 
detailed list of services displayed by 
date, program area and service type. 
Users can also view these services 
organized in a timeline so they can see 
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activity across multiple program areas 
summarized by month or year. 


Additionally, Client View provides access to 
electronic client documents such as petitions for 
dependency, school records and child permanency 
plans. It also provides access to client plans and 
assessment such as Family Plans, ANSA and CANS 
assessments. Having this information available to 
front-line staff eases the burden on clients to 
continually retell their story, and reduces 
inefficiencies and redundancies in the front-end information collection that intake and service staff 
must do. 

While the data contained in Client View are valuable for DHS and provider staff, DHS also recognizes 
that clients, and the families that support them, are often in the best position to coordinate their 
own care. For this reason, and to provide transparency around the information we collect, Client 
View also provides access for clients to view their own records and use that information to support 
their service needs and decisions. This access is 
built upon a partnership with the Commonwealth 
of Pennsylvania to link Client View with the client 
accounts for Pennsylvania's public benefits system, 

COMPASS. Because of this collaboration, DHS 
clients (many of whom also receive these public 
benefits) can use their existing COMPASS accounts 
(username and password) to also access Client 
View. They can get access to their records in Client 
View either directly from staff within DHS or from their service providers. A pin number and a 
document with directions on how to create and access the account is provided directly from within 
the Client View system. 

As of March 2018, the rollout of Client View to DHS's providers is complete, but additional sessions 
are still being scheduled by request. DHS has provided an hour-long in-person training and an 
electronic user guide to each provider. Most users, however, find the interface to be intuitive 
enough that using the system has not proved to be difficult. Both the documentation and the 
training provide guidance on how to give access to clients. Client View can provide a convenient 
first step to initiating a conversation about a client's service history, identifying potential service 
needs and strategizing next steps. 

Client View has been well-received by the provider agencies, and we have seen a lot of enthusiasm 
surrounding the use of the system and being able to incorporate this new tool into their business 
processes. 
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DHS has implemented some additional enhancements to Client View, which include the previously 
mentioned timeline view, as well as: 

• Improvements in the ability to search for clients and to search by client address 

• Ability to view emergency call data related to client addresses 

• Ability to connect with Pictometry data to view a location map and display oblique images of 
buildings and residences 

• Improved capabilities for DHS's Client View administrator, including the ability to conduct audit 
surveys and campaigns to verify and enforce appropriate use of the system 

• Greater capacity to download service records 

• Additional data sources including diagnoses, Health Choices eligibility and MA recipient 
information 

These enhancement, as well as many others, have been successfully incorporated into Client View 
and have been well received by internal staff and provider users. Additional changes being 
implemented include pulling in demographics from Health Department birth records and adding the 
district to the school records. 

Predictive Analytics : In August 2016, DHS implemented The Allegheny Family Screening Tool (AFST), 
a predictive risk model designed to improve call screening decision-making in the county's child 
welfare office. The AFST is the result of a two-year process of exploration, research, ethical analysis 
and training. 

DHS began the process with a Request for Proposals, through which a research team led by the 
Auckland University of Technology was selected to investigate opportunities for the use of 
predictive analytics in child welfare. 

The AFST calculates a Family Screening Score by 
integrating and analyzing hundreds of data elements on 
each person related to a child welfare referral. The 
score predicts the long-term likelihood of re-referral, if 
the referral is screened out without an investigation, or 
home removal, if the referral is screened in for 
investigation. If the Family Screening Score meets the 
threshold for "mandatory screen-in," the call must be 
investigated. In all other circumstances, the Family 
Screening Score provides additional information - in 
conjunction with clinical decision making - to assist call 
screening staff in making a call screening decision. 

Because It is only intended to inform call screening 
decisions and is not to be used in making investigative or other child welfare decisions, the score is 
available only to call screening staff. 

DHS published a report on the AFST and its development. This publication consists of three 
documents: 1) Developing Predictive Risk Models to Support Child Maltreatment Hotline Screening 
Decisions: Allegheny County Methodology and Implementation; 2) Ethical Analysis: Predictive Risk 
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Models at Call Screening for Allegheny County; and 3) DHS's response to the Ethical 
Analysis. Together, they provide a look at the development and implementation of the AFST. It can 
be found at Developing Predictive Risk Models to Support Child Maltreatment Hotline Screening 
Decisions . This publication also includes a Q&A document developed to provide answers to what we 
think will be the most common questions about the development and implementation of the AFST. 

2. How has the organization evaluated the program? Based on the evaluation, is DHS implementing 
any changes? 

Client View : We continue to evaluate the Client View application on an ongoing basis, making 
changes to functionality to meet the needs of internal users and providers. It is still early in the 
implementation to measure the level of adoption by providers. Based on our experience with the 
internal rollout and implementation of both Datavue (Client View's predecessor) and Client View, 
we know that integrating the use of similar systems into business processes often takes several 
years. We anticipate doing further targeted user satisfaction surveys to gather feedback on the 
utility, usability and functionality of the system and to identify areas for improvement. 

Predictive Analytics : Both a process and an impact evaluation were conducted. Through a Request 
for Proposals, Evaluation of a Predictive Risk Modeling Tool for Improving the Decisions of Child 
Welfare Workers. Hornby-Zeller Associates was selected to conduct the process evaluation and 
Stanford University to conduct the impact evaluation. These evaluations were funded by Casey 
Family Programs, the Human Services Integration Fund and DHS. 

The process evaluation was designed to assess the implementation process and the impact of 
screening scores on decision-making, through a combination of interviews, surveys and document 
reviews. Key questions included: 

• How has the tool modified the experience of call screeners? 

• What are the practice and policy implications? 

• What are perceptions and reactions to the tool? 

• How extensive were preparations for development of the model? 

The process evaluation report, which can be read at AFST Process Evaluation, found that: 

• The Predictive Risk Modeling Tool is facilitating data-driven decision-making with Allegheny 
County staff, but there is further room for system-level change. 

• Call screening staff report having a good understanding of the AFST, but are mixed on how 
confident they are in the resulting scores. 

• Call screen staff generally find the AFST easy to use, and offered technical suggestions for 
improving the Tool's user experience. 

The impact evaluation, which has been extended to include modifications made since 
implementation, will measure the impact of the AFST on: accuracy of decisions; reduction in 
unwarranted variation in decision-making; reduction in disparities and; overall referral rates and 
workload. It will assess the following outcomes: 

• Rate of screen in 
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• Likelihood of "bad" screen-outs 

• Likelihood of unnecessary screen-ins 

• Unwarranted variation in screening decisions 

• Disparity in screening decision 

The outcomes of these evaluations are being used to inform adjustments to the model. The 
evaluations and Frequently-Asked Questions about the evaluations will be posted on our website. 

3. How has the project impacted the strength and/or effectiveness of the organization? 

Both Client View and Predictive Analytics support DHS's mission of furthering data-driven decision 
making to improve the quality and impact of client services. With this grant, we made significant 
strides in how we use our integrated data to improve client services and expand access to 
information. Client View, by facilitating greater access to a person's complete service history, will 
improve the coordination of care for individuals within the human services system. The Allegheny 
Family Screening Tool helps DHS serve the County's children more effectively by providing call 
screeners with more information and helping them to make better informed clinical decisions about 
the disposition of referrals. 

Our use of predictive analytics has received a significant amount of attention, both locally and 
nationally. Below is a selection of links to existing coverage: 

• Actionable Intelligence 

This video, produced by University of Pennsylvania's Actionable Intelligence for Social Policy, 
highlights the Allegheny Family Screening Tool as one of the projects across the country that use 
integrated data systems (IDS) to improve social programs. 

• Can an Algorithm Tell When Kids Are in Danger? 

This New York Times article takes an in-depth look at Allegheny County's use of predictive 
analytics. This article also appeared in the Irish Examiner. 

• Can Big Data Help Save Abused Kids? 

This Reason article discusses efforts to use predictive analytics in preventing child abuse. 

• How Machine Learninfi Can Improve Public Sector Services 

Published by The Regulatory Review in October 2017, this article discusses how Allegheny 
County government digitized its records and uses big data analysis to improve health and human 
services. 

• Predictive Analytics in Child Welfare 

Published by the U.S. Department of Health and Human Services Office of the Assistance 
Secretary for Planning and Evaluation in November 2017, this report examines the potential 
benefits and pitfalls of predictive analytics and provides advice for progress. 

• Predictive Analytics in the Child Welfare System, Starting with the Basics 

Hosted by the Alliance for Racial Equity in Child Welfare and the Center for the Study of Social 
Policy. This webinar discusses the development and use of predictive analytics in child welfare. 
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• Using Integrated Data Systems (IDS) in County Government 
Four Annie E. Casey Foundation reports explore the value of using Integrated Data Systems to 
improve outcomes for individuals and families. 

• Pennsylvania County Leads in Use of Big Data to Stem Child Abuse. Probes Ethics First. The 
Chronicle of Social Change, May 26, 2017 

• Keeping kids safe: A smart new tool helps county see if a child is at risk . Pittsburgh Post-Gazette, 
April 13, 2017 

• Allegheny County DHS using algorithm to assist in child welfare screening. Pittsburgh Post- 
Gazette, April 10, 2017 

• Managing the Flow: Predictive Analytics in Child Welfare. Chronicle of Social Change, April 6, 
2017 

• Policy and Practice Changes Form Around National Strategy to Reduce Fatalities and Improve 
Child Safety, the Chronicle of Social Change, February 24, 2017 

• Predictive Analytics in the Child Welfare System, Starting with the Basics, a webinar hosted 
by the Alliance for Racial Equity in Child Welfare and the Center for the Study of Social 
Policy. 

4. What is the status of the project? 

For both projects, the technology portion requested in the grant has been completed, although 
implementation monitoring, continuous quality improvement and evaluation will continue. 

Client View : While the rollout and training for ClientView will likely diminish over the next few years, 
efforts to maintain and enhance ClientView will continue. Every time we make a change to the data 
warehouse, including adding a new source, we will update ClientView. 

Predictive Analytics : the AFST has been in operation since 2016 and will likely be ongoing for several 
years as we fine tune the modeling. 

5. What impact has this project had on the overall sustainability of the organization ? 

We anticipate that both projects will result in more effective allocation of resources, better use of 
data and improved client outcomes. 

6. Overall, how has the organization progressed toward long-term sustainability? 

Through technological innovations like Client View and the AFST, DHS continues to improve 
integrated, coordinated care for clients while focusing efforts and funds on the clients most in need. 


7. Audited financial statements for 2015: 

http://info.alleghenvcountv.us/WorkArea/DownloadAsset.aspx?id=5256 






I certify that this grant was used for the purpose designated in the grant application (Technology 
Improvements). 



Marc Cherna, Director 
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